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The University of Arkansas was 

founded in 1871 as the flagship institution of higher 
education for the state of Arkansas. Established as a 
land grant university, its mandate was threefold: to teach students, conduct research, and perform 
service and outreach. 



The College of Education and Health Professions established the Department of Education 
Reform in 2005. The department’s mission is to advance education and economic development 
by focusing on the improvement of academic achievement in elementary and secondary schools. 
It conducts research and demonstration projects in five primary areas of reform: teacher quality, 
leadership, policy, accountability, and school choice. 

The School Choice Demonstration Project (SCDP), based within the Department of Education 
Reform, is an education research center devoted to the non-partisan study of the effects of school 
choice policy and is staffed by leading school choice researchers and scholars. Led by Dr. Patrick 
J. Wolf, Professor of Education Reform and Endowed 21st Century Chair in School Choice, 
SCDP’s national team of researchers, institutional research partners and staff are devoted to the 
rigorous evaluation of school choice programs and other school improvement efforts across the 
country. The SCDP is committed to raising and advancing the public’s understanding of the 
strengths and limitations of school choice policies and programs by conducting comprehensive 
research on what happens to students, families, schools and communities when more parents are 
allowed to choose their child’s school. 
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Executive Summary 

In this report we examine high school completion and postsecondary enrollment (a.k.a. “educational 
attainment”) of the cohort of 9 th grade students who were in the Milwaukee Parental Choice Program 
(MPCP) at the beginning of our state- mandated evaluation of the MPCP in 2006. After tracking the 
MPCP 9 th graders following the 2006-07 year and comparing them to a carefully matched sample of 9 th 
graders who were in Milwaukee Public Schools (MPS) during the 2006-07 year, we use a combination 
of parent surveys and administrative (school) records to estimate attainment. We reached the following 
conclusions: 

• Overall, the primary finding of this report is that MPCP students had slightly higher rates of 
attainment than their MPS counterparts. This difference is primarily explained by the fact that more 
MPCP than MPS students both graduated from high school and enrolled in a four-year college. 
Some of the MPCP attainment benefit appears to be due to family background, as the attainment 
differences between our MPCP and MPS samples become smaller and lose statistical significance 
when we control for such factors as mother’s education, income, two-parent families, and religious 
attendance. 

• Ninth grade students who were in the MPCP in 2006-07 were more likely to graduate high school 
in 2009-2010 than similar 9th grade students who were in MPS in 2006-07. These differences 
persisted after accounting for race, gender and prior achievement, but the effects were not 
statistically significant. 

• MPCP students were more likely to have enrolled in a four year college, even after accounting for 
race, gender and prior achievement. They were less likely to have dropped out of high school or 
still be enrolled after four years. These differences may be partially explained by family background 
characteristics such as parental education and income. They do not appear to be related to private 
school “cream-skimming” of students into or out of the MPCP between 8 th and 9 th grade. 

• There was little difference between MPCP students and MPS students in attending a two-year or 
technical college. 

• Students in both sectors were far more likely to graduate and enroll in college if they remained in 
their initial sector (always in MPCP or always in MPS) from 2006-07 to 2009-10. This effect was 
stronger than any other attainment outcome we estimated, although it was particularly strong for 
MPCP students. 
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• MPCP and MPS dropouts shared the same primary explanations for leaving school: dislike of 
their school experience and academic difficulties. MPCP students were more likely to drop out 
due to teen pregnancy or mental health issues, whereas MPS students were more likely to drop 
out due to incarceration. 

We caution that these conclusions could be modified as we continue to follow these students through 
their fifth year following their entrance into 9 th grade in 2006 and add a second cohort of students 
(baseline 8th graders) to our analysis. 

This report and its companion reports continue a series of annual studies of the Milwaukee Parental 
Choice Program conducted by the School Choice Demonstration Project (SCDP).This ongoing 
research project is being funded by a diverse set of philanthropies including the Annie E. Casey, Joyce, 
Kern Family, Lynde and Harry Bradley, Robertson and Walton Family Foundations. We thank them 
for their generous support and acknowledge that the actual content of this report is the responsibility 
of the authors and does not reflect the official positions of the various funding organizations, 
the University of Arkansas, the University of Kentucky, Furman University, or the University of 
Wisconsin. We also express our deep gratitude to MPS, the private schools in the MPCP, and the 
state Department of Public Instruction for their willing cooperation, advice, and assistance. 
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1.) Introduction 

Policymakers and scholars alike have looked to studies of school choice programs for evidence that students 
do “better” or “worse” in alternatives to the traditional public sector. Nearly all of these studies have focused 
largely, if not entirely, on one particular outcome: the performance of students on standardized tests. Many 
of these studies acknowledge the importance of other outcomes. Some other outcomes have been studied, 
such as the indirect effects on other socially desirable goals like racial integration and the narrowing of 
racial, ethnic, and gender gaps in achievement (e.g. Greene, Mills and Buck 2010; Zimmer et al. 2009; 
Greene 2005; Neal 2006). Other outcomes such as the effects of school choice on student and parent 
satisfaction and civic values have also been analyzed in various studies (e.g. Greene and Forster 2003; 
Campbell 2008). 

As with other school choice programs, studies specifically of school voucher programs have primarily 
focused on student test scores. These include evaluations of privately funded programs (Howell et al. 2002), 
official analyses of public programs (Witte 2000; Wolf et al. 2010) as well as ongoing investigations such 
as the one directly tied to this report (Witte et al. 2008, 2009). Each of these studies has also reported on 
other outcomes to varying degrees, often finding large and positive voucher effects, while also reporting 
small or marginal effects on test scores. Positive voucher effects on parental satisfaction, sense of school 
safety, and civic values are prominent among these findings (e.g. Witte 2000; Howell et al. 2006; Wolf 
2007). 

Increasingly, analysts of school choice programs, including vouchers, are studying other outcomes besides 
test scores not simply because they represent alternatives to studying effects on educational quality, but 
because they represent fundamentally different measures of educational quality. Perhaps the most important 
of these is educational attainment: reaching a predefined level of schooling such as a high school diploma, 
enrollment in post-secondary education, or earning a bachelor’s degree and beyond. Several early studies 
examined the effect of attending a Catholic high school on student attainment (Coleman and Hoffer 1987; 
Neal 1997). These observational studies concluded that students graduated at much higher rates if they 
attended Catholic high schools, especially if they were urban minorities. Graduation and postsecondary 
enrollment is increasingly of interest in studies of other choice policies, most notably a multistate study of 
charter schools that found large attainment gains for students who moved from traditional public schools 
to charter schools (Zimmer et al. 2009). 

Although these findings of increased educational attainment from Catholic and charter schools are 
encouraging, school voucher programs allow students to attend a variety of private schools, not all of 
which will be Catholic. In the voucher literature, only two studies have examined the association between 
participating in a voucher program and graduating from high school. A recent experimental evaluation of 
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Washington, D.C.’s federal voucher program concluded that using a voucher increased the likelihood of high 
school graduation by 21 percentage points (Wolf et al. 2010). 1 

Educational attainment is an important indicator for school quality because it may be a direct result of the 
development of academic and life skills related to a variety of valuable outcomes of interest to policymakers 
and employers. These include regular employment, aversion to criminal and other dysfunctional behavior, and 
the generation and growth of personal income and savings. Studies have shown that students who have at least 
a high school degree can expect higher average life expectancy (Meara, Richards and Cutler 2008) and that 
even one-year increases in education can reduce the probability of dying in the next ten years (Lleras-Muney 
2005). College attainment is associated with higher levels of overall health (Wirt et al. 2004) and better health 
care (Muennig 2005; Rouse 2005). Not surprisingly, future wealth is also dependent on educational attainment 
(Rouse 2005; Caniero and Heckman 2003; Day and Newburger 2002), and this extends the benefits of higher 
attainment rates beyond the individual to broader social benefits such as increased tax revenue and economic 
development (Belfield and Levin 2007). Beyond pecuniary benefits, governments may see reductions in crime 
associated with increases in educational attainment (Belfield and Levin 2009; Levitt and Lochner 2001). 
Although such relationships between attainment and future success may not be surprising, graduation rates 
are still disturbingly low nationwide, especially for boys and particularly in the nation’s largest school districts 
(Greene and Winters 2006). 



Outcome 


Study 


Higher life expectancy 


Meara, Richards and Cutler (2008) 


Lower probability of death in near future 


Lleras-Muney (2005) 


Overall health and health care 


Wirt et al. (2004); Muennig (2005); Rouse (2005) 


Tax revenue and economic development 


Belfield and Levin (2007) 


Lower crime rates 


Belfield and Levin (2007); Levitt and Lochner (2001) 



Outcomes Associated with Higher Educational Attainment 



That Milwaukee is a large, urban school district only adds to the importance of the question of whether school 
choice boosts the levels of student attainment. If quality of life is directly related to educational attainment; if 
attainment is a direct result of certain schooling conditions to which a student is exposed; and if these schooling 



1 A recent observational study of the Milwaukee Parental Choice Program concluded that voucher students in seven of the 26 
participating private high schools graduated at rates 1 2 percentage points higher than their MPS counterparts in both 2007-08 
and 2008-09 (Warren 2011). That study was unable to acquire administrative lists of graduates from MPCP schools or conduct 
parental surveys so the data limitations were considerable. The author notes that he awaits our more comprehensive study. 
Our findings are in the same direction, but the differences we report are considerably lower. 
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conditions may vary as a result of individual parent and student decisions, then the long-term social and 
economic consequences of school choice programs may be far greater than the impact of such policies on more 
transitory outcomes like individual test scores. In this report we provide evidence that attainment may indeed be 
related to the school choices families make, at least insofar as these choices pertain to a voucher-funded private 
or traditional public school. We first present basic tabulations of high school graduation and two and four-year 
enrollment as they vary by sector. Next, we consider these differences after accounting for student background. 
We then estimate a statistical model that predicts the overall level of attainment students achieved after four 
years, and consider other explanations for the results we obtain. We conclude by discussing these results in the 
context of ongoing and future research on public-private differences in student outcomes. 

2.) Data 

The sample for this study includes 801 MPCP students and 801 MPS students who were in 9 th grade during the 
2006-07 academic year. The 801 MPCP students are the entire 9th grade cohort of students who we determined 
to be valid voucher-using students after examining the Wisconsin Department of Public Instruction audited list 
of voucher recipients based on the 3 rd Friday count (September 15, 2006). The MPS students are, on the other 
hand, a sample of the population of 9 th graders in MPS during the fall of 2006. They are not a random sample, 
but instead are a group of students who we carefully matched to the MPCP population of 9th graders on the 
basis of several important characteristics. 

2 A.) The Matching Algorithm: Addressing Observed and Unobserved Student Characteristics Associated 
with MPCP Enrollment 

The match between MPCP and MPS students was critical for reasons outlined in Witte et al. (2008, 2009). 
Briefly, neither we nor other researchers evaluating school choice programs believe that students who select 
alternatives to the public sector do so for non-random reasons. If these non-random reasons are also related to 
the outcome of interest then any differences attributed to the impact of the choice program could be biased. In 
the case of this study, we were particularly worried that students who chose to participate in the MPCP in 9 th 
grade may be more likely to graduate high school and enroll in college naturally, regardless of the school they 
attend. Such factors could be un-measureable and therefore threaten to bias the analysis. 

There are a variety of statistical approaches that researchers take to address such problems. The matching 
algorithm we employed is a multi-stage process that selected MPS 9 th graders who were nearly identical to 
MPCP students with respect to several key characteristics. In the first stage we matched students on their home 
neighborhoods in Milwaukee. We did this in sequence for each student in the sample of MPCP 9th grade 
students. Following the advice of demographers and city planners, we used census tracts to identify student 
neighborhoods. Census tracts are drawn by the U.S. Census Bureau to follow neighborhood boundaries. In 
our sample, MPCP students come from 175 different census tracts. In this stage, for any given MPCP student 
in our sample, we restricted the list of potential MPS matches to students in the same grade and tract. We 
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prioritized a tract match because we believe that students’ initial neighborhoods will serve as a control for a 
number of unobserved variables that may affect outcomes, including future educational attainment. 2 

In the second stage, we matched students in their census tracts who were within the same 5 th percentile 
bandwidth of Benchmark test scores. We matched students in our longitudinal panel in grades 3-8 using the 
Wisconsin Knowledge and Concept Exam (WKCE), which 9 th graders in Wisconsin do not take. However, 9 th 
grade MPS students do sit for the Benchmark exam, which we obtained from the MPS district to administer to 
9th grade MPCP panelists in November 2006, when their counterparts in MPS were sitting for that test as well. 

In the third stage of our match, if more than one MPS student was matched to the MPCP student based on 
census tract and test scores, or if there were missing values for either variable for an MPCP student, we matched 
by estimating propensity scores (Rosenbaum and Rubin 1983). The function of the propensity score is to identify 
MPS students with characteristics that are typical of MPCP students and therefore signal their “propensity” to 
be in the MPCP even though they are not. In this step, we estimated the propensity of MPCP participation as 
a function of the mean of math and reading Benchmark scores, gender, race and an indicator for students with 
English Language Learning status. The MPS student with the closest propensity score to the MPCP student 
was then selected. If missing predictors made it impossible to predict a propensity score for the MPCP student, 
the MPS student was selected at random from MPS students remaining in the running after matching on 
census tract and prior test. If the missing predictor was student test score, matches were made at random within 
tract. The Witte et al. (2008) report describes in detail the success of this match. Briefly, all matched students 
fell within a tenth of a standard deviation on math test scores, and within less than one-hundredth of a standard 
deviation in reading scores. No statistical differences in race, gender or English learning status were evident. The 
two groups are very similar to each other in important ways, by design. Survey data taken after the first year of 
testing indicated that the two groups were highly similar in many additional family background characteristics 
that were not and could not be used for the match, although MPCP parents indicated more frequent religious 
attendance. 

2B.) Obtaining 2009-2010 Attainment Status for 2006-07 9 th Graders 

After the initial 9 th grade match in 2006, we tracked students into the following year when most students 
entered 10 th grade and were therefore due to take a final WKCE test. We discerned no major achievement 
differences between the MPCP and MPS respondents in our study (Witte et al. 2009). We did not track these 
baseline 9th grade students in 2008-09, when they would not have taken the WKCE, but did so again in 2009- 
2010, the year they were due to enter 12 th grade and, at its end, complete high school. We used two primary 



2 Evidence for neighborhood effects on social outcomes is presented across several social science disciplines. See, for example, 
Aaronson (1998) for evidence of neighborhood effects on educational outcomes even after family characteristics are taken 
into account; Ludwig, Ladd and Duncan (2001) and Leventhal and Brooks-Gunn (2004) for experimental evidence linking 
neighborhood improvements to improvements in student outcomes; and Sampson, Morenoff and Gannon-Rowley (2002) for a 
general discussion. See also Cullen, Jacob and Leavitt (2005) for use of census tract information in research on school choice. 
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sources of information to determine student graduation status. Hie first was a set of administrative files, and the 
second was a detailed survey we administered to the parents of MPS and MPCP students who were in 9 th grade 
in 2006. 

The administrative files came from two sources. The first was a graduation list and a supplemental end-of- 
year enrollment status file from MPS dated after the 2009-2010 school year (July 2010). The second was 
a list of 2010 graduates from each of the participating MPCP high schools. We examined both lists for all 
1,602 students in our study who were in 9 th grade in 2006. Specifically, we checked both the MPCP and MPS 
graduation lists for the original 801 MPCP panelists and checked both the MPS and MPCP graduation lists 
for the 801 MPS panelists. A student who started out as a 9th grader in the MPCP could easily have graduated 
from MPS, and vice-versa. The operation of the school choice program, specifically the paucity of high schools 
in the program, makes such transfers common (Cowen et al. 2010). 

These sources, while valuable for confirming graduation status and current enrollment, did not provide us with 
other pieces of information about attainment, notably enrollment in postsecondary education, and they did not 
provide detailed explanations for the failure to graduate on time. For this information, we attempted to contact 
parents of all 1,602 panelists via a telephone survey in the summer of 2010. We received responses from 61.3 
percent (491/801) of the original MPCP panelists and 62.6 percent (501/801) of the original MPS panelists. 
These are very high response rates for populations of students in urban areas, particularly for families of students 
who entered the analysis via a procedure that took place four years earlier. Students did not vary by race among 
respondents and non-respondents. The respondents were slightly more likely to be female, and had higher 
Benchmark scores in 2006, than non-respondents. In the analysis below, we use response weights to correct for 
any baseline differences. 

Table 1 indicates the sources of information on students’ graduation status after the 2009-2010 year, by initial 
status. The single largest set of students was the most important: those for whom graduation was confirmed by 
both our survey and through administrative sources. The next largest categories were students who appeared 
to have graduated in administrative records but did not respond to the survey, and students whose parents 
indicated graduation in the survey but could not be found in MPS and MPCP enrollment databases. The 
remaining categories describe various small groups of students: those who did not graduate but did not respond 
to the survey, and so on. The most important implication of Table 1 is that for students for whom we have both 
survey and administrative data , there is remarkable consistency in graduation indicators. Less than one percent 
of students in each sector were considered graduates in administrative records but non-graduates in the survey 
(group 5 — highlighted in the table), and approximately one percent in each sector had such an inconsistency 
in reverse: non-graduates in administrative records but graduates in the survey (group 6 — highlighted in the 
table). This remarkably high level of consistency between the survey and administrative data for students with 
records in both sets of data is critical because it allows us to base our analysis primarily on the survey records, 
which, as we describe below, contain most of the information necessary for this study. Our study is the first 
school choice analysis to establish the consistency of parent reports of educational attainment with actual 
administrative records. 



Student Attainment and the Milwaukee Parental Choice Program 



6 


March 2011 







Table 1: Administrative and Survey Student Status Information 2009-2010 



Group 


Administrative Record Says 


Survey Says 


MPS in 
2006 (%) 


MPCP in 
2006 (%) 


1 


Enrolled/Did Not Graduate 


Did Not Graduate 


8.0 


3.5 


2 


Graduated 


Graduated 


37.6 


33.0 


3 


Graduated 


No response 


16.9 


14.7 


4 


Withdrawn/Not Enrolled 


Graduated 


4.5 


12.7 


5 


Graduated 


Did Not Graduate 


0.5 


0.8 


6 


Enrolled/Did Not Graduate 


Graduated 


1.3 


0.3 


7 


Withdrawn/Not Enrolled 


Earned a GED 


1.0 


0.8 


8 


Enrolled/Did Not Graduate 


No response 


6.2 


3.9 


9 


Withdrawn/Not Enrolled 


Did not graduate (still in 
high school) 


4.7 


7.1 


10 


Withdrawn/Not Enrolled 


Did not graduate (notin 
high school) 


5.0 


3.3 


11 


Withdrawn/Not Enrolled 


No response 


14.4 


20.1 




Total (%) 
N 




100.0 

801 


100.0 

801 



NOTES: "Withdrawn/Not-Enrolled" categories are from MPS files; they confirm non-graduation in MPS but do not confirm non- 
graduation elsewhere, and we treat as analogous to non-response in the survey. Groups 5 and 6 are highlighted because they 
represent conflicting graduation information for students for whom we have both survey and administrative records. Sources: 
Milwaukee Public Schools enrollment database as of fall 2010; Official 2010 graduation lists of all private high schools participating in 
the Milwaukee Parental Choice Program; Parent telephone survey regarding student status administered in the summer of 2010. 



3.) On-Time Graduation and College Enrollment Rates 

Table 2 presents our estimated confirmed high school graduation rates using all sources of information per Table 
1, as well as those based only on our survey respondents. This table is based on the initial status of panelists 
during our 2006 baseline. These rates are calculated excluding unknowns from the denominator. 3 The MPS rate 
of 69.3 percent is well within the range reported in an official MPS analysis of student attainment released in 
2009 based on earlier cohorts of students (Carl et al. 2009). The MPCP rate is higher than the MPS rate, at 
75 to 77 percent, depending on the source of information. Of the non-graduates, some may still be enrolled 
in school — these would be students who take longer than the expected four years to graduate — or they may 



3 If unknowns were to be included, the rates would obviously be lower, but this would be tantamount to assuming that all 

unknowns did not graduate. If a greater percentage of unknowns graduated than knowns, our reported rates are too low. If the 
reverse, our rates are too high. For comparisons between sectors to be biased, one would have to assume that more unknowns 
graduated in one sector than the other. We have no evidence that is true. 
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have dropped out. 4 We consider these students in greater detail below. Finally, although our focus here is on 
differences between MPCP and MPS on average, readers may note that there are differences within each sector 
as well. See Figure Al in Appendix A for the distribution of graduation rates across schools serving students in 
our sample. 



Table 2: Confirmed 2009-10 On-Time Graduation Rate 





MPS in 2006 (%) 


MPCP in 2006 (%) 


Graduated: 






All Sources 


69.4 

(476/686 records) 


76.6 

(490/640 records) 


Survey Respondents Only 


69.3 

(347/501 respondents) 


75.0 

(368/491 respondents) 



Sources: Milwaukee Public Schools enrollment database as of fall 2010; Official 2010 graduation lists of all private high schools 
participating in the Milwaukee Parental Choice Program; Parent telephone survey regarding student status administered in the summer 
of 2010. 

The sample design that we described above minimizes bias when comparing graduation rates by initial status. 
This comparison has a somewhat restrictive policy interpretation, however, because it means that the results 
in Table 2 indicate that the graduation rate for students who were in MPCP as 9 th graders is higher than 
for students who were in MPS as 9 th graders. Although we believe that this is the only unbiased comparison 
available in the data, we recognize that many readers will be interested in information on students who remain 
in MPCP or remain in MPS for more than just their freshman year. To that end, we calculate the confirmed 
graduation rate for students who stayed in and those who left their initial sector at some point after 2006, based 
on our tracking results during the second and fourth years after the 2006 baseline. As Table 3 indicates, the 
graduation rates for students who stayed in their initial status, regardless of sector, were much higher than for 
those who left — albeit the difference is much greater for MPCP. Of the 2006 9 th grade students who stayed 
in the voucher schools for four years, 94 percent graduated. The rate for students who stayed in MPS, while 
lower (75 percent), is comparable to the graduation rate for students who were in MPCP in 2006 regardless of 
whether the latter stayed in the same sector (e.g. the rates reported in Table 2). 

The graduation rates in Table 3 are calculated using all sources, in part to maximize our ability to determine 
student sector location. However, because of the nearly perfect consistency between the administrative records 
and the survey records for students located in both files, we are comfortable using the graduation response 
rate from the survey data as our measure to proceed further. We do so because much of the remaining data are 
only available for survey respondents. The first of these is an indication of postsecondary plans. Table 4 reports 



4 They may also have completed a GED, although given the short period of time between the end of the school year (June 2010) 
and our surveys (mid-summer 2010), this is highly unlikely. Only 1 percent of respondents indicated that they had already 
received a GED. 
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technical, two- and four-year college enrollment rates by initial status and by location in the same sector over 
time. As with high school graduation, the rate of enrollment in a four-year college is higher for students who 
were 9 th graders in MPCP (38%) than for MPS 9 th graders (31%), and about 20 percentage points higher for 
students who remained in the MPCP during all four years of high school. This very large difference coincides 
with the result above for graduating from high school. The differences between rates of two-year enrollment 
and technical college enrollment between the MPCP and MPS students are quite small, but the rates are higher 
for MPS in both categories. Combined this suggests that MPCP graduates were somewhat more likely to select 
four-year colleges while MPS students tended to chose technical or two-year college alternatives. 

Table 3: Confirmed 2009-10 Graduate Rate By Sector Location, based on Survey and Administrative Sources 





MPS in 2006 (%) 


MPCP in 2006 (%) 


Confirmed Same Sector 2006-2009 


74.8 

(421/563 records) 


94.3 

(300/318 records) 


Left Sector 2006-2009 


44.7 

(55/123 records) 


59.0 

(190/322 records) 


Overall (by Initial Sector) 


69.4 

(476/686 records) 


76.6 

(490/640 records) 



NOTES: Sector locations estimated after tracking 2006-07 9th graders after the 2006, 2007 and 2009 academic years. Sources for 
sector of high school enrollment: Milwaukee Public Schools official enrollment files, 2006-2009; Enrollment confirmations from private 
schools in the Milwaukee Parental Choice Program, 2006-2009; Project-initiated telephone calls to parents, 2006-2009. Source for 
graduation rates: Milwaukee Public Schools enrollment database as of fall 2010; Official 2010 graduation lists of all private high schools 
participating in the Milwaukee Parental Choice Program; Parent telephone survey regarding student status administered in the summer 
of 2010. 



Table 4: Technical, Two and Four-Year College Enrollment Rates After 2009-201 0 Academic Year 





MPS in 2006 


MPCP in 2006 


Technical 
School (%) 


Two-Year 
College (%) 


Four-Year 
College (%) 


Technical 
School (%) 


Two-Year 
College (%) 


Four-Year 
College (%) 


Confirmed Same 
Sector 2006-2009 


13.8 

(54/391 

records) 


16.4 

(64/391 

records) 


34.5 

(135/391 

records) 


11.2 

(27/241 

records) 


12.5 

(30/241 

records) 


54.4 

(131/241 

records) 


Left Sector 2006- 
2009 


10.9 

(12/110 

records) 


10.0 

(11/110 

records) 


18.2 

(20/110 

records) 


14.0 

(35/250 

records) 


11.6 

(29/250 

records) 


21.2 

(54/250 

records) 


Overall (by Initial 
Sector) 


13.1 

(66/501 

records) 


14.9 

(75/501 

records) 


30.9 

(155/501 

records) 


12.6 

(62/491 

records) 


12.0 

(59/491 

records) 


37.7 

(185/491 

records) 



Sources for sector of high school enrollment: Milwaukee Public Schools official enrollment files, 2006-2009; Enrollment confirmations 
from private schools in the Milwaukee Parental Choice Program, 2006-2009; Project-initiated telephone calls to parents, 2006-2009. 
Source for post-secondary enrollment: parent telephone survey regarding student status administered in the summer of 2010. 
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4.) Predicting Individual Student Attainment 

In the preceding section, we focused on between-sector differences in attainment rates — the percentage of 
students overall who did or did not graduate, and did or did not sign-up for postsecondary education. These are 
important indicators for policymakers concerned with a host of large-scale decisions ranging from education 
budgets in the short-run to producing a well-educated citizenry in the long-run. On the other hand, these 
differences provide only limited information on what characteristics predict graduation, or postsecondary 
enrollment, or any other attainment indicator for the individual student. Certainly the results above strongly 
suggest that exposure to the MPCP increases the likelihood that a student graduates, but other factors such 
as race, gender, and academic ability can influence graduation rates as well. Importantly, as we have noted, we 
believe that our matching algorithm ensures that the sector differences above are unbiased estimates of actual 
attainment differences. Thus we expect that any other student characteristics influencing attainment are not 
systematically related to the initial sector in which a student was located for purposes of our study. 



4 A.) Predicting Graduation and Postsecondary Enrollment Separately 

High School Graduation . Our basic approach to understanding how the voucher program affects individual 
students is to use a multivariate analysis to estimate the likelihood a student in the MPCP will graduate 
at a higher or lower rate than a student in our matched-MPS sample. We will first account for student 
characteristics, then also include a control for prior (2006) achievement, then control for if a student was in the 
same sector (MPCP or MPS) for all four years, and then include all possible predictive independent variables: 
sector, student characteristics, prior achievement, and staying in the same sector. For clarity we provide the exact 
statistical models. 



We begin by estimating the probability, P, that a given student, i, graduated from high school in 2009-2010 as 
the logit function with the general form: 



where 



P(graduate) = 7 

1 + e~ Zl 



Eq. (1) Z = (3 0 + fi l MPCP06. + (3,Race ( + fifiender. 



In Equation 1 , (3 ] is the effect that initial status in MPCP (in 2006) had on the probability that a student 
graduated from high school in 2009-2010, after accounting for the student’s race and gender, both of which are 
common control variables in models of high school success. 5 

We also estimate 

Eq. (2) Z - |3 0 + (1 1 M PCP06 i + (yRacc + |3 3 Gender + |3 4 9f/z Grade Test. 



5 See Appendix A for statistics on model covariates. 
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where 9 th grade achievement is measured as the average of a student’s 2006 Benchmark math and reading scores 
(standardized to have a mean of zero and a standard deviation of one). This is an important characteristic 
because it is possible that, despite our matching procedure in 2006, the MPCP effect on graduation may be 
explained by differences in baseline student achievement in that sector. This would be the case if, for example, 
the MPCP difference is explained by a few very high performing students who are more likely to go to college 
than their public school counterparts. 

Table 5 provides the estimates of Equations 1 and 2, and these indicate that the effect of the MPCP is 
statistically insignificant at conventional levels, with standard errors clustered by school. We estimate the 
marginal effect of the MPCP to be a 5 percent increase in the probability of graduation. This is 1 to 2 percentage 
points lower than the simple mean difference in graduation rates between the MPCP and MPS students 
discussed above. This implies that, although race, gender and prior levels of achievement do not explain most of 
the MPCP effect, per se, including these characteristics in a prediction of MPCP impacts on attainment does 
not allow us to reject the possibility that there is no MPCP effect with traditional levels of confidence. 6 This is 
not an issue of bias associated with these observed variables: that the marginal effects in Table 5 are similar to 
the mean differences in Tables 2 and 3 indicates that the MPCP-MPS difference is not primarily explained by 
race, gender or prior achievement. It is, however, an issue of collinearity: including these variables in a model of 
graduation reduces our ability to differentiate the individual effects associated with each variable. 



6 There are other student characteristics that researchers often consider when studying student success, but which may not 
be directly comparable between MPCP and MPS: socioeconomic status, for example, or special educational needs (ExEd).The 
former is often measured by indicating whether a student participates in a federal free/reduced lunch program (FRL). As we 
note in earlier reports, however, (Witte et al. 2008, 2009, 2010), these characteristics may be measured quite differently in MPS, 
which is required by law to provide a free/reduced lunch program and provide support for special needs children, whereas no 
such requirements exist in the MPCP. A student who is not flagged as FRL in MPCP may be eligible/participant in MPS, and vice 
versa. More problematic is the ExEd distinction. Many schools in MPCP do not have exceptional needs programs, and students 
who are not flagged as ExEd in MPCP may actually be undiagnosed ExEd students, or maybe ExEd students receiving their 
program at MPS while attending the rest of school in MPCP. For these reasons, we do not have reliable data for MPCP students 
for FRL and ExEd categories. We do believe, however, that we have captured the underlying dynamic of SES and ExEd in these 
results. For SES, MPCP students are eligible for the program specifically because their income is tied to federal FRL guidelines — 
these are all comparably low-income students. Moreover the fact that neighborhood is taken into account should also account 
for any large-scale differences in socioeconomic status. In addition, we believe that the inclusion of student prior achievement 
should generally account for ExEd. For MPS students (those for whom we have a reliable ExEd measure), students flagged as 
ExEd have an average Benchmark score that is four-tenths of a standard deviation below the citywide mean. 
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Table 5: Predicting High School Graduation After Four Years 





(1) 


(2) 


MPCP in 2006 


0.26 


0.28 




(0.21) 


(0.21) 


Black 


-0.81** 


-0.31 




(0.35) 


(0.31) 


Hispanic 


-0.66* 


-0.11 




(0.34) 


(0.39) 


Asian 


2.15* 


2.14** 




(1.06) 


(1.07) 


Female 


0.74*** 


0.50*** 




(0.14) 


(0.15) 


Mean 2006 Benchmark 


— 


0.71*** 


(standardized) 




(0.15) 


Constant 


1.11*** 


0.84** 




(0.36) 


(0.32) 


N 


992 


837 


Estimated 


0.05 


0.05 


MPCP Marginal Effect 


(0.04) 


(0.04) 



NOTES: ***p<0.01; **p<0.05; *p<0.10, two-tailed, based on standard errors reported in parentheses, clustered by baseline school. 
Marginal effects are calculated at the mean of other variables. Responses are weighted to account for survey non-response and 
missing 2006 achievement data. Sources for sector and demographics are Milwaukee Public Schools official enrollment files, 2006- 
2009 and enrollment confirmations from private schools in the Milwaukee Parental Choice Program, 2006-2009. Source for Benchmark 
achievement are MPS test files (MPS) and project-administered examinations in 2006. Source for graduation rates are for project- 
administered parent telephone survey regarding student status administered in the summer of 2010. See Table 1 for cross-tabulations 
of survey responses to graduation data from administrative sources where available. 



Post-Secondary Enrollment . All of the issues discussed above with regard to high school graduation are 
germane to predicting postsecondary enrollment. Rather than restate them here, we refer to our general logit 
equation, where P is now defined as enrolling in a given type of postsecondary institution, and in Equations 1 
and 2, the same control variables predicting graduation are now used to predict postsecondary enrollment. For 
simplicity, we combine enrollment in a technical school or a two-year school into one category. Table 6 displays 
estimates of Equations 1 and 2 for the probability of enrolling in a technical/two-year school and a four-year 
institution, respectively. As in Table 4, there is no MPCP difference in the probability that a student attends a 
technical or two-year college. Consistent with Table 4, however, there is a positive MPCP effect on enrolling in 
a four-year school, and this effect appears to be similar in size to the MPCP effect on high school graduation. 
As in the prediction of graduation, these estimates are not statistically significant at conventional levels. Also, as 
explained above, this insignificance is not a matter of bias but of collinearity between the variables in the model. 
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Table 6: Predicting College Enrollment After Four Years 





Two-Year or Technical College 


Four-Year College 




(1) 


(2) 


(1) 


(2) 


MPCP in 2006 


-0.13 


-0.09 


0.27 


0.30 




(0.16) 


(0.19) 


(0.22) 


(0.20) 


Black 


0.40 


0.69 


-0.64** 


-0.31 




(0.40) 


(0.42) 


(0.28) 


(0.29) 


Hispanic 


0.31 


0.57 


-0.59* 


-0.35 




(0.46) 


(0.47) 


(0.33) 


(0.31) 


Asian 


0.18 


0.34 


-0.01 


0.04 




(0.75) 


(0.82) 


(0.48) 


(0.48) 


Female 


-0.19 


-0.31* 


0.87*** 


0.81*** 




(0.15) 


(0.16) 


(0.15) 


(0.18) 


Mean 2006 Benchmark 


— 


0.03 


— 


0.66*** 


(standardized) 




(0.12) 




(0.17) 


Constant 


-1.20*** 


-1.39** 


-0.76*** 


-1.07*** 




(0.36) 


(0.39) 


(0.34) 


(0.32) 


N 


988 


835 


992 


837 












Estimated 


-0.03 


-0.02 


0.05 


0.06 


MPCP Marginal Effect 


(0.04) 


(0.04) 


(0.04) 


(0.04) 



NOTES: ***p<0.01; **p<0.05; *p<0.10, two-tailed, based on standard errors reported in parentheses, clustered by baseline school. 
Marginal effects are calculated at the mean of other variables. Responses are weighted to account for survey non-response and 
missing 2006 achievement data. Sources for sector and demographics are Milwaukee Public Schools official enrollment files, 2006- 
2009 and enrollment confirmations from private schools in the Milwaukee Parental Choice Program, 2006-2009. Source for Benchmark 
achievement are MPS test files (MPS) and project-administered examinations in 2006. Source for graduation rates are for project- 
administered parent telephone survey regarding student status administered in the summer of 2010. See Table 1 for cross-tabulations 
of survey responses to graduation data from administrative sources where available. 



4B.) Predicting Overall Level of Attainment 

We now turn to the main results of this report. In the preceding section we considered the effect of the MPCP 
on the probability that a student achieves a certain dichotomous designation: finishing high school (yes or 
no) and enrolling in a college (yes or no). Here we consider how enrollment in the MPCP affects the overall 
level of a student’s attainment in 2010. This is distinct from the previous question because there is no overlap. 
Every student in our analysis had a particular level of “highest educational attainment” by the summer of 2010. 
Policymakers may wish to consider the impact of MPCP on the probability that a student drops out of high 
school instead of staying in school and graduating “late,” for example, or the probability of enrolling in college 
instead of graduating from high school and not pursuing post-secondary education (PSE). We consider five 
distinct attainment levels, j: 1 = dropping out of high school; 2 = still in high school; 3 = high school graduation, 
but no post-secondary education; 4 = enrollment in technical or two-year PSE; and 5 = enrollment in a four- 
year institution. Table 7 displays summaries of these categories by sector. The important point evident here is 
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that overall attainment is generally similar in both MPS and MPCP at each level, with the greatest difference 
coming at the top level: enrollment in a four-year college. 7 Thus the slight differences in graduation above are 
largely shown here to consist of students who graduated high school and enrolled in college. We now consider 
whether this difference persists after other factors are taken into account. 



Table 7: Reported Overall Attainment Levels After Four Years 



Attainment Level After 2009-2010 School Year 


MPS in 2006 


MPCP in 2006 


High School Dropout 


9.6 


6.5 


Still in High School 


20.4 


17.3 


Graduated High School, No PSE 


15.0 


17.7 


Two-Year or Technical College 


24.6 


21.4 


Four- Year College 


30.5 


37.1 


Total 


100.0 


100.0 




N=501 


N=491 



SOURCE: Parent telephone survey regarding student status administered in the summer of 2010. 

We focus here on the marginal effect of the MPCP on the probability that a student’s attainment level is one 
of the j categories instead of another. Table 8 presents the estimated ordered probit coefficient for MPCP 
enrollment, which is positive and statistically significant at p < 0.10, indicating that being enrolled in the MPCP 
in 9th grade has a positive effect on achieving relatively higher levels on the attainment scale (see Appendix 
B for more detail on the ordered probit results). The marginal effects of the MPCP on the likelihood of each 
level of attainment further indicate that MPCP students are less likely to be at the lower end of the attainment 
scale (e.g. high school dropout or still in school) and more likely to be at the higher end than are initial MPS 
panelists. The marginal effect on attending a four-year school is striking, consistent with the previous logit 
model simply predicting college enrollment: the effect of the MPCP on the probability of attending a four- 
year college, an increase of almost 7 percentage points, is more than 20 percent of the size of the overall mean 
enrollment rate. 



7 There is a slight difference between two year/technical enrollment rates between Table 7 and Table 4 for MPS. This is due to a 
small number of students who reported plans to enroll in some form of technical program but had not reported graduating. In 
Table 4 these are considered enrollees, in Table 7, which is concerned explicitly with levels of attainment they are considered 
non-graduates. 
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Table 8: Estimated MPCP Effects on Attainment Levels After Four Years 





MPCP in 2006 


Marg 


inal Effect of MPCP on Probability 


ir of Attainment Level 




Coef. (s.e.) 


Dropout 


Still in HS 


Graduate HS 
Only 


Two-Year 

College 


Four-Year 

College 


Estimate 

(s.e.) 


0.18* 

(0.10) 


-0.02 

(0.01) 


-0.04* 

(0.02) 


-0.02** 

(0.01) 


0.00 

(0.00) 


0.07* 

(0.04) 



NOTES: ***p<0.01; **p<0.05; *p<0.10, two-tailed, based on standard errors reported in parentheses, clustered by baseline school. 
Marginal effects are calculated at the mean of other variables. Responses are weighted to account for survey non-response and 
missing 2006 achievement data. Sources for sector and demographics are Milwaukee Public Schools official enrollment files, 2006- 
2009 and enrollment confirmations from private schools in the Milwaukee Parental Choice Program, 2006-2009. Source for Benchmark 
achievement are MPS test files (MPS) and project-administered examinations in 2006. Source for graduation rates are for project- 
administered parent telephone survey regarding student status administered in the summer of 2010. See Table 1 for cross-tabulations 
of survey responses to graduation data from administrative sources where available. 



5.) Other Explanations 

The analysis above considers the effect of attending MPCP in 9 th grade on future graduation and post-secondary 
enrollment. Whether this effect can be considered an answer to a causal question — were students more likely 
to attend college because they enrolled in the MPCP in 9 th grade? — depends on the extent to which our analysis 
accounts for other factors that made students both a.) more likely to attend MPCP in 9 th grade and b.) more 
likely to graduate high school or attend college in the first place. As in many other evaluations of such programs, 
such factors may be largely unobservable or immeasurable to the analyst. 

The matching algorithm we describe above is intended to eliminate many of these unobserved factors, because 
it accounts not only for student achievement, but also student demographics and students’ initial neighborhood 
location in 2006. As we have noted, there is a considerable body of academic research that has demonstrated 
the relationship between neighborhood location and unobserved family characteristics. But it is possible that 
even this match fails to account for the sort of attributes leading some parents to send their children to MPCP 
schools and also relating to future attainment. 

5 A.) Family Characteristics 

Although we cannot address this possibility for the full sample of students analyzed above, a sub-sample of 
these students had parents who responded not only to our attainment survey in 2010, but also to a baseline 
characteristics survey in 2006-07, 2007-08 and 2008-09. The earlier survey included information on family 
income, whether the student had both parents in the home, religious attendance and parental education levels. 
Each of these measures has been shown in a variety of education studies to be related to student outcomes. In 
particular, we should certainly expect that having a parent graduate from high school or college would improve a 
student’s chances of reaching that goal herself. 

Table 9 presents the same information as Table 8, but these new estimates are adjusted for income, parental 
education, whether a student had both parents at home, and religious attendance. See Appendix B for full 
results, which also demonstrate that parental education is indeed related in this sample to overall attainment 
levels. Table 9 indicates that the MPCP effect is still positive, but is no longer statistically significant. This 
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is partly due to the reduced sample size, so any estimates become less precise. However, the evidence on 
this sub-sample does indicate that the family background factors may explain some although not all of the 
MPCP effect. The reason is that the coefficient and the marginal effect in Table 9 are noticeably lower than 
their corresponding values in Table 8, whereas in the analysis above the inclusion of race, gender and prior 
achievement in estimation of a particular educational attainment, such as high school graduation, did not 
generally change the MPCP-MPS difference. 

Figure 1 depicts the MPCP difference in overall attainment graphically by plotting the predicted probabilities 
for a student in each category based on whether they were in the MPCP or the MPS in 9th grade. The 
probabilities in the solid lines are based on Table 8, and are calculated after adjusting for the covariates in 
Equation 2, with each covariate held constant at its mean. Similarly, the probabilities in the dotted lines are 
based on Table 9, with the survey covariates added. There are two important implications of Figure 1. The first 
is that, regardless of model, the greatest difference between 9th grade students who were in MPCP in 2006 
and those who were in MPS in 2006 is that the MPCP students are particularly more likely to graduate high 
school on time and enroll in college. That is indicated by the differences on the far right in four-year college 
between either the two solid lines or the two dotted lines. The differences are between 5% and 6% higher for 
MPCP students. The negative (1 percentage point) difference in the likelihood that MPCP students graduate 
and go no further is significant at conventional levels, but is so small that MPCP and MPS students appear 
essentially equally likely to graduate and go no further. The second implication is that accounting for the family 
background characteristics does not change this story. The major difference between the solid and dotted lines is 
a slight increase (from 32 to 34 percent) in the probability of attending college for original MPS panelists. 



Table 9: Estimated MPCP Effects on Attainment Levels After Year One Accounting for Parental Income, 
Education, Marital Status and Religious Attendance 





MPCP in 2006 


Marg 


inal Effect of MPCP on Probability of Attainment Level 




Coef. (s.e.) 


Dropout 


Still in HS 


Graduate HS 
Only 


Two-Year 

College 


Four-Year 

College 


Estimate 


0.10 


- 0.01 


- 0.02 


- 0.01 


0.00 


0.04 


(s.e.) 


( 0 . 13 ) 


( 0 . 01 ) 


( 0 . 03 ) 


( 0 . 01 ) 


( 0 . 00 ) 


( 0 . 05 ) 



NOTES: ***p<0.01; **p<0.05; *p<0.10, two-tailed, based on standard errors reported in parentheses, clustered by baseline school. 
Marginal effects calculated at the mean of other variables. Responses weighted to account for survey non-response and missing 
2006 achievement data. Sources for sector and demographics are Milwaukee Public Schools official enrollment files, 2006-2009 
and enrollment confirmations from private schools in the Milwaukee Parental Choice Program, 2006-2009; Source for Benchmark 
achievement are MPS test files (MPS) and project-administered examinations in 2006. Source for graduation rates are for project- 
administered parent telephone survey regarding student status administered in the summer of 2010. Source for parental education, 
income, marital status and religious attendance is project-administered surveys in 2006 and 2007. See Table 1 for cross-tabulations 
of survey responses to graduation data from administrative sources where available. See Appendix B for full model results on survey 
covariates. 
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NOTES: Probabilities based on probit estimates of the various categories of attainment, calculated with all other variables in the model 
held constant at their means. Sources for sector and demographics are Milwaukee Public Schools official enrollment files, 2006-2009 and 
enrollment confirmations from private schools in the Milwaukee Parental Choice Program, 2006-2009; Source for Benchmark achievement 
are MPS test files (MPS) and project-administered examinations in 2006. Source for graduation rates are for project-administered parent 
telephone survey regarding student status administered in the summer of 2010. See Table 1 for cross-tabulations of survey responses to 
graduation data from administrative sources where available. 



5B.) Selection into 9 th Grade Private Schools 

Compared to students in 8 th grade or below, there are fewer high school students in the MPCP. The 801 9 th grade 
panelists we examined in 2006 were all the ninth graders in the MPCP that year. In total, the students in the 
MPCP numbered more than 17,000 that year. Less than one-third of more than 110 MPCP schools served high 
school students in 2009-10. These small numbers could exacerbate the selection bias problems described above, if 
students who stay in the MPCP for high school are doing so specifically to increase their attainment chances, or if 
MPCP schools implicitly or explicitly select the better students. The latter possibility is frequently raised in academic 
research if private schools can “counsel out” or even expel students that public schools cannot, or if 8 th graders in 
voucher programs must apply to highly selective college preparatory high schools. Although we cannot test these 
possibilities directly, we can consider a likely result of such forms of “cream-skimming:” we can test if students who 
leave the MPCP in 8 th grade are systematically different from students who stay into the 9 th grade. 

We do not have 8 th grade data on the 9 th graders under study in this report, but we do have 8 th and 9 th grade data on 
students who were in earlier grades when our study began in 2006. We have no reason to believe that these three 9 th 
grade cohorts are systematically different from the earlier 2006-07 cohort whose attainment we study in this report, 
although we will examine this possibility next year. For the purposes of brevity, we do not consider all of the possible 
characteristics of students in the body of this report, but we refer interested readers to Appendix B, Table B-3 for 
details. In sum, we find no systematic evidence that students who remain in the MPCP for 9 th grade are dramatically 
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different in terms of demographics, prior achievement, parental education, income, marital status or religiosity 
than MPCP students who switch to the MPS for high school. If there is an unobservable selection effect driving 
the MPCP differences noted above, it does not appear to be related to a host of student characteristics we would 
expect to be relevant if MPCP schools were selectively enrolling the better students, or displacing other students 
between 8 th and 9 th grade. In short, for cohorts following the one in this paper, there appears to be no cream 
skimming of 8 th graders as they enter high school. 

SC.) Remaining in MPCP for the Duration of High School 

The estimated impacts in Equations 1 and 2 are the effects of being in the MPCP in 2006-7 on the probability 
of graduating in 2010-11. As Table 3 shows, however, students moved out of MPS and the MPCP during 
this time period, and their graduation rates were different depending on whether they made such a move. In 
particular, the graduation rate among students who stayed in the MPCP, 94 percent, is much higher than the 
rate for any other group. Our matching algorithm was designed to address only the correlation between a 
student’s 2006 sector of enrollment (MPCP or MPS) and unobserved variables. We cannot simply compare the 
group of students who stayed in the MPCP for four years to those who did not — students who stayed in the 
MPCP may have been more likely to graduate in the first place, for reasons unaccounted for in our matching 
algorithm (and unrelated to the variables in Equations 1 and 2). Although we cannot measure these reasons, we 
can measure whether a student stayed in their initial sector and then estimate the effect of being in MPCP with 
a control for this sector persistence: 

Eq. (3) Z = |3 0 + (3 { MPCP06 . + |3 2 Race. + (3 ^Gender. + ['> 4 9th Grade Test + (3 5 SameSector. 

In such a model, where Z is graduating from high school or not, |3j is the effect of being in the MPCP in 2006 
after accounting for any influences on the probability of graduating that are common to students who stayed in 
their respective sectors (fi 5 SameSectori). 

Table 10 presents estimates of Equations 1 and 2 for high school graduation, with a control for staying in the 
same sector included. As expected by the results in Table 3, the estimates of f'y are much higher than in the 
earlier specifications. From these we calculate the marginal effect of remaining in the MPCP for all four years 
of high school: an increase in the likelihood of graduation of about 18 percentage points when controlling for 
race and gender and almost 14 percentage points when also controlling for baseline achievement on the 9 th 
grade Benchmark test. Students who began and stayed in the MPCP throughout their high school experience 
graduated at much higher levels than did MPS students, all else equal. These results are similar when the 
outcome is post-secondary enrollment. 8 



8 Specifically, the marginal effects for enrolling in college, paralleling the last two rows in Table 10, were for MPCP, .14 without 
baseline test score, and .13 with test score included. The overall effect of staying in the same sector was .26 without baseline 
score and .24 with it included. All of these effects were significant at p<.01 . As with high school graduation this means that 
being in MPCP for four years led to a 13% higher probability of enrolling in college than for students in MPS. For both groups, 
staying in the same sector for four years led to a 24% greater chance of enrolling in college. These estimates include a control 
on baseline achievement in 2006. 
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Regardless of where students began their high school careers, remaining in the same sector had a positive 
impact on graduation. The marginal effect of this impact (the last row in Table 10) is more than 30 percentage 
points: larger than any of the estimated MPCP effects we report here. Thus, what this supplementary analysis 
finds is that staying in one sector for four years is likely to produce on-time graduation. And this result is more 
pronounced for private schools. 



Table 1 0: Predicting High School Graduation After Four Years, Controlling for Sector Status 





(1) 


(2) 


MPCP in 2006 


0.96*** 

(0.23) 


0.83*** 

(0.24) 


Black 


-0.77** 

(0.36) 


-0.30 

(0.36) 


Hispanic 


-0.79** 

(0.34) 


-0.23 

(0.41) 


Asian 


2.17** 

(1.09) 


2.23** 

(1.13) 


Female 


0.67*** 

(0.16) 


0.48*** 

(0.17) 


Mean 2006 Benchmark (standardized) 


— 


0.65*** 

(0.12) 


Confirmed Same Sector 2006-08 


1.85*** 

(0.21) 


1.67*** 

(0.24) 


Constant 


-0.24 

(0.42) 


-0.42 

(0.40) 


N 


992 


837 








Estimated MPCP Marginal Effect 


0.18*** 

(0.04) 


0.14*** 

(0.04) 


Estimated Same Sector Marginal Effect 


0.36*** 

(0.04) 


0.32*** 

(0.04) 



NOTES: ***p<0.01; **p<0.05; *p<0.10, two-tailed, based on standard errors reported in parentheses, clustered by baseline school. 
Marginal effects calculated at the mean of other variables. Responses weighted to account for survey non-response and missing 
2006 achievement data. Sources for sector and demographics are Milwaukee Public Schools official enrollment files, 2006-2009 
and enrollment confirmations from private schools in the Milwaukee Parental Choice Program, 2006-2009; Source for Benchmark 
achievement are MPS test files (MPS) and project-administered examinations in 2006. Source for graduation rates are for project- 
administered parent telephone survey regarding student status administered in the summer of 2010. See Table 1 for cross-tabulations 
of survey responses to graduation data from administrative sources where available. 
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6.) Why Did Students Not Graduate? 

In the final component to this analysis, we consider explanations for students who did not graduate. These 
students fall into two general categories: those who did not graduate because they were no longer enrolled in 
school, and those who did not graduate because they were still enrolled. These correspond to the first two rows 
of Table 7. Dropout rates are of particular interest to researchers and policymakers. We estimate the dropout rate 
across the four years of our study at 6.5 percent for MPCP initial 9th graders compared to 9.6 percent for initial 
MPS 9th grade students. Our estimated MPS dropout rate is slightly higher than the most recent dropout rate 
reported by MPS for 9 th - 12 th graders of 8.0 percent for the 2007-08 year. 9 Twenty percent of our original MPS 
panelists did not graduate but were classified as still in school, while 17 percent of our original MPCP panelists 
did not graduate but were classified as still in school. 

Unfortunately, the survey data do not explore in detail the different reasons that students did not graduate but 
remained in school. They either were retained in one or more of their high school grades or they dropped out 
for a period of time, but we have no information regarding why they were retained or dropped out. We did, 
however, consider the group of confirmed high school dropouts as of the end of the 2009-10 school year. 

Table 1 1 reports the results of several questions to which parents of 68 high school dropouts responded. We 
intended their answers to provide explanations for why their children left high school. Answer categories were 
not mutually exclusive, and parents could cite more than one. As Table 1 1 indicates, by far the most common 
explanation for dropping out in both sectors of original panelists was that their child did not like school, 
especially MPCP students. Similar answers — that students disliked a new school or were bored in school — were 
less common but still relatively prominent among the explanations. These are general, somewhat ambiguous 
answers because the reasons why a child did not like school (or was bored) could be myriad, ranging from an 
overall perception that school was a waste of time to problems with teachers or peers. A second potential reason 
for disliking school could be genuine difficulty understanding the coursework. An inability to keep up with 
school work was indeed a relatively frequent response for both sectors, as was the notion that obtaining a GED 
would be easier. In the sample, only 9 survey respondents among MPS panelists (1.8 percent) and 6 MPCP 
respondents (1.2 percent) had actually obtained a GED. 



9 MPS District Report Card, 2008-09. http://www2.milwaukee.k12.wi.us/acctrep/0809/2009_district.pdf. We do not view this 
slight discrepancy as problematic. It could be a result of the fact that our sample deliberately matched to a slightly lower- 
scoring population of MPS students than the population of students overall; it could be an artifact of the calculation we made 
based on the survey reports, whereas MPS simply divides the number of withdrawn 9-1 2 th graders by the total number of 
enrollees; it could be caused by the fact that our rate is calculated on the same panel of students observed over time, whereas 
MPS's rate is calculated on a cross-section of students each year; or it could simply be the result of a small measurement error 
inherent in any estimate. 
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Table 11 : Reasons For Leaving High School Prior to Graduation 





MPS in 2006 (%) 


MPCP in 2006 (%) 


"My child didn't like school"or"My child didn't get along with 
his/her teachers" 


31.8 


51.3 


"My child thought it would be easier to get a GED" 


27.5 


24.8 


"My child was bored in school" 


16.2 


20.2 


"My child was suspended or expelled from school" 


16.2 


10.1 


"My child could not keep up with the schoolwork" 


15.3 


10.1 


"My child changed schools and didn't like the new school" 


10.7 


10.4 


"My child needed to take care of or support his/her family" 


7.8 


7.0 


"Child incarcerated" 


oo 

r-C 


0.0 


"My child didn't get along with other students" 


5.2 


0.0 


"Child in Mexico" 


5.2 


0.0 


"Child is/was pregnant"or"Child got married and had a baby," 
or"Child has mental health issues" 


5.0 


13.5 


"My child didn't feel safe at school" 


0.0 


0.0 



NOTES: Figures based on the 68 survey respondents whose child left high school between 2006-07 and 2009-10. Categories are not 
mutually exclusive (respondents could give more than one answer) and statistics are weighted for survey non-response. Source: 
parent telephone survey regarding student status administered in the summer of 2010. 

Behavioral problems appear to have been more prevalent explanations for dropping out among the original 
MPS panelists. Sixteen percent dropped out because they were expelled or suspended, and nearly 8 percent 
because they were incarcerated. Only 10 percent of MPCP panelists were expelled or suspended, and none 
were incarcerated. It is difficult to directly compare the expulsion/suspension figures between the two sectors, 
however, because expulsion/suspension is a formal process within MPS — one that may involve legal implications 
as well. On the MPCP side, while some students could be formally removed from their school, they may also 
be “counseled out” or “asked not to return” in lieu of such a formal measure. Previous reports (e.g. Cowen et al. 
2010) have indicated that these explanations were among the reasons why students left their original MPCP 
schools (regardless of whether they actually dropped out of school entirely). 

Similar numbers of students dropped out to care for their families — about 7 to 8 percent in each sector. On the 
other hand, 13.2 percent of MPCP panelists dropped out because they became pregnant, had a baby, or had 
mental health issues, compared to 5 percent of MPS panelists. There are several reasons why this might be the 
case. We do not know whether more MPCP panelists actually had babies — only that more dropped out because 
they were expecting. Some MPS schools may provide resources to students with children, thus reducing the 
need to drop out. There is evidence in other studies (e.g., Figlio and Ludwig 2000) that students in Catholic 
schools — the single most common type of MPCP school — are more likely to become teenage parents, possibly 
for reasons related to the Catholic Church’s doctrinal positions on contraception and abortion. Perhaps this is 
the case here as well. 
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Overall, it appears that the most important reasons for dropping out of high school are relatively common in 
both sectors. MPS students were more likely to list incarceration and MPCP students were more likely to list 
pregnancy or mental health issues. In both sectors, dropping out was most clearly related to a general dislike of 
school, and/or problems that could be related to learning difficulties or even underlying ability. This explanation 
was particularly common on the MPCP side. These explanations should surprise neither policymakers nor 
educators. If there is anything unexpected in these results, it is the fact that the dominant reasons for dropping 
out are similar in both sectors. 



7.) Summary 

In this report we analyzed data on all students confirmed to be 9 th graders in the Milwaukee Parental Choice 
Program during the 2006-07 academic year as well as a carefully matched comparison sample of 9th graders in 
Milwaukee Public Schools that same year. For those students following normal or “on-time” progress through an 
American high school, the 2009-2010 academic year would have corresponded to their final year in high school. 
After tracking both sets of students in their second and fourth years following the 2006-07 baseline year, we 
reached the following conclusions: 

• Overall, the primary finding of this report is that MPCP students had slightly higher rates of attainment 
than their MPS counterparts. This difference is primarily explained by the fact that more MPCP than 
MPS students both graduated from high school and enrolled in a four-year college. Some of the MPCP 
attainment benefit appears to be due to family background, as the attainment differences between our 
MPCP and MPS samples become smaller and lose statistical significance when we control for such 
factors as mother’s education, income, two-parent families, and religious attendance. 

• Ninth grade students who were in the MPCP in 2006-07 were more likely to graduate high school in 
2009-2010 than similar 9th grade students who were in MPS in 2006-07. These differences persisted after 
accounting for race, gender and prior achievement, but the effects were not statistically significant. 

• MPCP students were more likely to have enrolled in a four year college, even after accounting for race, 
gender and prior achievement. They were less likely to have dropped out of high school or remained 
enrolled after four years. These differences may be partially explained by family background characteristics 
such as parental education and income. They do not appear to be related to private school “cream- 
skimming” of students into or out of MPCP between 8 th and 9 th grade. 

• There was little difference between MPCP students and MPS students in attending a two-year or 
technical college. 

• Students in both sectors were far more likely to graduate and enroll in college if they remained in their 
initial sector (always in MPCP or always in MPS) from 2006-07 to 2009-10. This effect was stronger than 
any other attainment outcome we estimated, although it was particularly strong for MPCP students. 

• MPCP and MPS dropouts shared the same primary explanations for leaving school: dislike of their school 
experience and academic difficulties. MPCP students were more likely to drop out due to teen pregnancy 
or mental health issues, whereas MPS students were more likely to drop out due to incarceration. 
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As with any evaluation, there are caveats to these findings. The first of which is that, except when otherwise 
noted, any effects we are describing pertain to attendance in MPCP in 2006. The results should be interpreted 
as the effect of “exposure” to the MPCP during high school rather than long-term persistence in that sector. 
Although we have included an estimate of the effect of MPCP after accounting for whether students remained 
in that sector for their entire high school experience, we caution that our study design does not ensure that such 
a calculation accounts for all explanations for why students may have remained in the MPCP after we matched 
them to public school students four years ago. Nevertheless, we note that 94 percent of MPCP students who 
stayed in the MPCP from 9th through 12th grade graduated high school on time, and more than half enrolled 
in a four-year college. 

The second caveat is that we have yet to consider what it means to be educated in the MPCP as opposed to 
MPS. Forthcoming analyses will explore in detail the school contexts in which MPCP and MPS students are 
learning, including the possibility that certain school curricula, activities or practices are particularly associated 
with later student success in remaining engaged in the educational process. Consistent with our evaluation 
of MPCP effects on student achievement (test scores), we do not analyze differences in graduation or college 
enrollment that are associated with particular MPCP schools. Previous academic research on urban education 
has suggested that school quality is especially variable in such a context. Although we note in Appendix B that 
there are school-based differences in graduation for students in our sample, we leave to future scheduled work the 
sort of in-depth examination of these schools to explain them. In this report we can simply say that, on average, 
there are modest MPCP-MPS attainment differences that appear to favor the MPCP. 

Our ability to estimate precisely these differences depends on the number of other factors that we take into 
account. As we have noted, for race, gender, and prior achievement this is not an issue of bias, as the point 
estimates of the MPCP difference do not dramatically change when these factors are considered. There is a 
reduction, however, (e.g. from a 7 to a 4 point MPCP effect) when we add parental education, income, marital 
status, and religiosity, so it is possible that these differences account for some of the positive MPCP effect 
on educational attainment. This change is subject to the same lack of precision as in the models above, so we 
caution against drawing a firm conclusion on the exact size of the effect of the MPCP. 

The preponderance of the evidence here suggests that there is a modest positive impact of the MPCP on student 
educational attainment. Though related to, it is not entirely explained by race, gender, prior achievement, 
parental education, income, religiosity, or neighborhood location. These results are broadly consistent with 
research in other cities and other school choice programs (Wolf et al. 2010; Zimmer et al. 2009) as well as 
Warren’s (2011) descriptive study of high school graduation rates in the MPCP and MPS. This positive effect 
of school choice on educational attainment is apparent in places where students who chose an alternative to 
traditional public schooling do not necessarily appear to be performing significantly better on other measures 
such as standardized test scores (e.g., Wolf et al. 2010). Likewise, the fourth-year report in our longitudinal 
evaluation of student achievement (Witte et al. 2011) finds no major differences in achievement test scores 
between MPCP students and similar MPS students. 

Thus if the results we present here are interpreted as evidence that MPCP students are performing slightly 
better on one metric — attaining a given level of education — they do not support a comprehensive conclusion 



Student Attainment and the Milwaukee Parental Choice Program 



March 2011 


23 







that the MPCP necessarily provides a better learning environment than MPS. Although we believe that high 
school graduation and college enrollment are critically important, we suggest that readers who are interested 
in such a comprehensive conclusion will be best served by considering the evidence in this report alongside the 
other analyses we have released both this year and in prior years of this comprehensive longitudinal evaluation. 10 
Finally, future research will follow a second cohort of students through four year graduation (8 th graders 
in 2006), and the remaining 2006 9 th graders who may become fifth-year graduates. Those results will be 
reported next year. 



10 All 28 reports to date can be accessed at http://www.uark.edu/ua/der/SCDP/Milwaukee_Research.html 
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APPENDIX A: 



Covariates and 201 0 Graduation Survey Response Information: 



Table A-1 : Statistics on Model Covariates 





MPS in 2006 


MPCP in 2006 


Black 


0.72 


0.72 


Hispanic 


0.16 


0.17 


Asian 


0.04 


0.03 


White 


0.07 


0.07 


Female 


0.53 


0.59** 


Mean 2006 Benchmark (standardized) 


0.06 


0.12 



NOTES: **Indicates statistically different from MPS at p<0.05. Statistics are weighted for survey non-response for consistency with 
model estimation, although race and gender based on 801 MPS students and MPCP students; Benchmark scores are average of math 
and reading scores standardized to MPS mean, and based on 750 MPS students and 583 MPCP students, respectively. Sources for 
sector and demographic are Milwaukee Public Schools official enrollment files, 2006-2009 and enrollment confirmations from private 
schools in the Milwaukee Parental Choice Program, 2006-2009; Source for Benchmark achievement are MPS test files (MPS) and 
project-administered examinations in 2006. Source for graduation rates are for project-administered parent telephone survey regarding 
student status administered in the summer of 2010. See Table 1 for cross-tabulations of survey responses to graduation data from 
administrative sources where available. 



Table A-2 Response Difference by Sector and Student Characteristic 





No Survey Response 


Survey Response 


Logit Coefficient 


MPCP in 2006 


0.51 


0.49 


-0.05 (0.10) 


Black 


0.75 


0.70* 


-0.43 (0.20)* 


Hispanic 


0.16 


0.17 


-0.34(0.23) 


Asian 


0.03 


0.03 


-0.11 (0.34) 


White 


0.05 


0.08* 


(ref category) 


Female 


0.56 


0.56 


0.01 (0.10) 


Constant 


- 


- 


0.88 (0.21)* 


N 


610 


992 


1602 



*Significantly different from non-respondents at p<0.05 





MPS in 2006 


MPCP in 2006 


No Survey Response 


Survey Response 


No Survey Response 


Survey Response 


Black 


0.73 


0.71 


0.76 


0.71* 


White 


0.05 


0.08* 


0.06 


0.07 


N 


300 


501 


310 


491 



*Significant!y different from non-respondents at p<0.05 
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Percentile of School Graduation Rate 



Rates calculated for 26 MPCP schools and 48 MPS schools with students in our sample. These are not school-wide estimates, but are 
obtained by dividing the number of sample students indicating high school graduation by the number of sample students linked to each 
school. This table should be interpreted as follows: for example, the 25th percentile of school graduation rates is 67 percent for MPCP, 
and 43 percent for MPS. The 90th percentile is 88 percent graduation for both MPCP and MPS. 
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APPENDIX B: Other Model Results 

Ordered Probit Estimates of Overall Attainment 

To obtain the effects of the MPCP on a student’s probability of each category, we estimate the ordered probit: 

Eq.(Bl) P(j= 1) = 4>(t i - Z ; ) 

P(j = 2) = 4>(t 2 - Z.) - 4>(t, - Z.) 

P(j = 3) = <J>(t 3 - Z.) - cj)(t 2 - Z.) 

P(j = 4) = (J)(X 4 - z.) - (J>(T 3 - Z.) 

P(j = 5) = 4>(t 5 - Z.) - 4>(t 4 - Z.) 



Where Z. is the same set of relationships described in Equation 2, which represents what we consider the most 
empirically sound model we have available because it includes prior achievement. 



Table B-1 Ordered Probit Estimates of Equation A1 (Overall Attainment) 





Coefficient 


MPCP in 2006 


0.18** 

(0.08) 


Black 


-0.02 

(0.14) 


Hispanic 


-0.02 

(0.15) 


Asian 


0.32* 

(0.23) 


Female 


0.36*** 

(0.07) 


Mean 2006 Benchmark (standardized) 


0.40*** 

(0.07) 


T i 


_1 ^3*** 

(0.17) 


E 


-0.34** 

(0.17) 




0.09 

(0.16) 


T 4 


0.74*** 

(0.17) 


N 


837 



***p <0 01 ; **p<0.05; *p<0.10, two-tailed, based on standard errors reported in parentheses, clustered by baseline school Sources 
for sector and demographic are Milwaukee Public Schools official enrollment files, 2006-2009 and enrollment confirmations from 
private schools in the Milwaukee Parental Choice Program, 2006-2009; Source for Benchmark achievement are MPS test files (MPS) 
and project-administered examinations in 2006. Source for graduation rates are for project-administered parent telephone survey 
regarding student status administered in the summer of 2010. See Table 1 for cross-tabulations of survey responses to graduation 
data from administrative sources where available. 
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Table B-2: Ordered Probit Estimates of Equation A-1 (Overall Attainment) After Accounting for Parental Income, 

Education, Marital Status and Religious Attendance 





No Survey Variables 


Survey Variables 


MPCPin 2006 


0.14 


0.10 




( 0 . 12 ) 


( 0 . 13 ) 


Black 


- 0.10 


- 0.08 




( 0 . 16 ) 


( 0 . 19 ) 


Hispanic 


0.02 


0.04 




( 0 . 16 ) 


( 0 . 19 ) 


Asian 


0.08 


0.19 




( 0 . 22 ) 


( 0 . 24 ) 


Female 


0 . 48 *** 


0 . 49 *** 




( 0 . 09 ) 


( 0 . 09 ) 


Mean 2006 Benchmark (standardized) 


0 . 36 *** 


0 . 33 *** 




( 0 . 08 ) 


( 0 . 08 ) 


Income >$50K 




0.00 




- 


( 0 . 18 ) 


Income $35-50K 




0.19 




- 


( 0 . 14 ) 


Income $25-35K 




0.11 




- 


( 0 . 14 ) 


Parent attainment: college 




0 . 69 *** 




- 


( 0 . 23 ) 


Parent attainment: high school 




0.08 




- 


( 0 . 14 ) 


Parent attainment: some college 




0.23 




- 


( 0 . 14 ) 


Both parents living at home 




0 . 18 * 




- 


( 0 . 11 ) 


Attends regular religious service 




- 0.09 




- 


( 0 . 11 ) 


Does not attend religious service 




- 0 . 23 * 




- 


( 0 . 14 ) 




_1 


_*| -|*** 




( 0 . 18 ) 


( 0 . 24 ) 


h 


- 0 . 40 ** 


- 0 . 19 ** 




( 0 . 16 ) 


( 0 . 23 ) 
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T 3 


0.06 


0.28 


(0.17) 


(0.22) 


T 4 


0.69*** 


0.94*** 


(0.17) 


(0.21) 


N 


603 


603 



***p<0.01; **p<0.05; *p<0.10, two-tailed, based on standard errors reported in parentheses, clustered by baseline school Sources 
for sector and demographic are Milwaukee Public Schools official enrollment files, 2006-2009 and enrollment confirmations from 
private schools in the Milwaukee Parental Choice Program, 2006-2009; Source for Benchmark achievement are MPS test files (MPS) 
and project-administered examinations in 2006. Source for graduation rates are for project-administered parent telephone survey 
regarding student status administered in the summer of 2010. Source for parental education, income, marital status and religious 
attendance is project-administered surveys in 2006 and 2007. See Table 1 for cross-tabulations of survey responses to graduation data 
from administrative sources where available. 



Analysis of Students Moving from 8 th Grade in MPCP to 9 th Grade in MPCP 

As noted in the text, we do not observe 8 th grade data on students who were in 9 th grade in 2006-07, the first 
year of our study However, we do have data on subsequent cohorts of students (those who were in 8 th grade in 
2006-07, 8 th grade in 2007-08, or 8 th grade in 2008-09. From this information, and using the results of survey 
data obtained in 2006-07, we are able to estimate the probability of remaining in private school for original 
MPCP panelists. 

Table B3: Predicting 9 th Grade Private School Enrollment by 8 th Grade MPCP Students 





Estimates (s.e.) 


Black 


-0.52 

(0.42) 


Hispanic 


0.05 

(0.47) 


Asian 


-0.69 

(1.07) 


Female 


0.48** 

(0.22) 


WKCE Math (8 th grade) 


-0.08 

(0.18) 


WKCE Reading (8 th grade) 


0.23 

(0.17) 


Income >$50K 


-0.47 

(0.49) 


Income $35-50K 


0.63* 

(0.36) 


Income $25-35K 


0.20 

(0.29) 
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Parent attainment: college 


0.23 

( 0 . 41 ) 


Parent attainment: high school 


- 0.42 

( 0 . 32 ) 


Parent attainment: some college 


- 0.08 

( 0 . 32 ) 


Both parents living at home 


- 0.20 

( 0 . 27 ) 


Attends regular religious service 


0.14 

( 0 . 24 ) 


Does not attend religious service 


- 0.63 

( 0 . 47 ) 


Constant 


0.36 

( 0 . 51 ) 


N 


395 



***p<0.01; **p<0.05; *p<0.10, two-tailed, based on standard errors reported in parentheses. Sources for sector and demographic 
are Milwaukee Public Schools official enrollment files, 2006-2009 and enrollment confirmations from private schools in the Milwaukee 
Parental Choice Program, 2006-2009; Source for WKCE exams are MPS test files (MPS) and project-administered examinations in 
2006. Source for graduation rates are for project-administered parent telephone survey regarding student status administered in the 
summer of 2010. Source for parental education, income, marital status and religious attendance is project-administered surveys in 
2006 and 2007. See Table 1 for cross-tabulations of survey responses to graduation data from administrative sources where available 
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